Beyond the Death of Linear Response: 1// optimal information transport. 
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Non-ergodic renewal processes have recently been shown by several authors to be insensitive to 
periodic perturbations, thereby apparently sanctioning the death of linear response, a building block 
of nonequilibrium statistical physics. We show that it is possible to go beyond the "death of linear 
response" and establish a permanent correlation between an external stimulus and the response of 
a complex network generating non-ergodic renewal processes, by taking as stimulus a similar non- 
ergodic process. The ideal condition of l//-noise corresponds to a singularity that is expected to 
be relevant in several experimental conditions . 
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I 

o 



> 

O 

o 



X 



There has been a surge of interest in understanding 
the dynamics of complex netviforks over the past decade 
with studies ranging from the topology of transportation 
webs, to the connectivity of communication meshes to 
the dynamics of neuron networks. Most recently the im- 
portance of the nascent theory of information exchange 
between complex networks has become evident. 

In living neural networks the connection between func- 
tion and information transport is studied with experi- 
mental techniques of increasing efficiency [1] from which 
an attractive perspective is emerging, i.e. these complex 
networks live in a state of phase transition (collective, 
cooperative behavior), a critical condition that has the 
effect of optimizing information transmission j^]. From 
the studies of complex networks it is evident that the sta- 
tistical distributions for network properties are inverse 
power laws and that the power-law index is a measure 
of the degree of complexity. Intimate connections ex- 
ist between neural organization and information theory, 
the empirical laws of perception pi] and the production 
of 1// noise [J|, with the surprising property that 1// 
signals are encoded and transmitted by sensory neurons 
with higher efficiency than white noise signals [5|. Al- 
though 1// noise production is interpreted bypsycholo- 
gists as a manifestation of human cognition [6|, and by 
neuro-physiologists [7] as a sign of neural activity, a the- 
ory explaining why this form of noise is important for 
communication purposes does not exist yet. 

The well known stochastic resonance phenomenon [8| 
describes the transport of information through a random 
medium, obeying the prescriptions of Kubo Linear Re- 
sponse Theory (LRT) j9| , being consequently limited [10| 
to the stationary equilibrium condition. There are many 
complex networks that generate 1// noise and violate 
this condition: two relevant examples are blinking quan- 
tum dots Hi and liquid crystals [l2|.The non-Poisson 
nature of the renewal processes generated in these ex- 
amples jl3| is accompanied by ergodicity breakdown and 



non-stationary behavior 14 1. 

The response of non-stationary networks to harmonic 
perturbation has recently been found by many authors 
|15l42C| to fade away with time, an effect called death 
of linear response [15]. This result seems to call into 
question one of the fundamental theories of statistical 
physics, that being the fluctuation-dissipation theorem 
and the resulting LRT of Kubo [9|.The intuitive expla- 
nation of this effect is as follows. The complex network 
is prepared and perturbed at time i = 0. Experimen- 
tal preparation generates a cascade of events, whose rate 
R{t) is a decreasing function of time, thereby making the 
response, which is proportional to R{t), fade away. Is this 
a general property, independent of whether the stimulus 
is periodic [15-20]? If this was a general result, it would 
be difficult, if not impossible, to explain the communica- 
tion properties revealed by the recent neuro-physiological 
literature. 

The purpose of this Letter is to prove that the death 
of LRT actually rests on an extension of Kubo LRT to 
the non-stationary condition (NSLRT) and that conse- 
quently a non-ergodic system, insensitive to perturba- 
tions with a fixed time scale, does respond to perturba- 
tions sharing the same non-ergodic behavior. We shall 
argue that this important phenomenon explains why 1// 
noise is an efficient stimulus for complex systems. 

The NSLRT rests on the general LRT form 
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where the subscripts S and P denote the "system" net- 
work and the perturbing network, respectively. Note 
that ('Cs(i)) is the Gibbs ensemble average over infinitely 
many responses Cs(0 to ^p{t) and e ^ 1 is the stimulus 
strength. We make the simplifying but realistic assump- 
tion that the preparation of S [12| does not set a bias on 
S, so that (^s(O)) — 0. The function x(^i *) is given by 
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The function ^s(i,i') is the autocorrelation function of 
^s(t), namely, the survival probability of age i', and 
Rs{t) for the case of discrete signals considered here, 
is the rate at which events are produced by the net- 
work S prepared at t = 0, i.e. the bits per second in- 
coded in S,s{t). This rate is time independent only in the 
Poisson case. In the non-Poisson case this rate depends 
on time, thereby making ^^(i,^') non-stationary. The 
brand new survival probability ^s(0 — ^s(^i^'=0)i is 
given by [H, m Q 



^s{t)^{l+t/Ts) 



l-A^s 
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from which the corresponding waiting-times probability 
density V's(^) = —d^s{t)/dt is derived. In the range of 
parameters 1 < /is < 3 considered here, it is known f28'| 
that: 
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for 1 < fis < 2 (4) 
for 2 < /is < 3, (5) 



with Ts — Ts/{ns " 2) the mean value of 'ipsi't)- 

When /is < 2 the experimental preparation of S in- 
duces a sequence of events, whose rate Rs tends to van- 
ish for t — >■ oo, yielding a perennial out-of-equilibrium 
condition, and an explanation of the death of linear re- 
sponse |15l420| as well. In fact, the response to a har- 
monic perturbation of frequency / is proportional to 
l/(/t)2-A's [i^. In the case 2 < /is < 3, on the con- 
trary, the preparation-induced cascade of events, in the 
limit t — ^ oo, becomes stationary and virtually identical 
to that of a Poisson process. The theoretical analysis of 
this Letter is done in the asymptotic time regime. Thus, 
we refer to the case 2 < /i < 3 as stationary, in contrast 
to the non- stationary case /i < 2 of perennial transition. 
Before proceeding with the use of the NSLRT of Eq. ([T|) , 
we point out some important properties of both signals 
£,s{t) and ^p{t). If necessary, the signal £,p{t) must share 
the same properties as ^s{t) and for simplicity they are 
both assumed to be dichotomous signals with random re- 
newal fluctuations between the values -1-1 and —1. The 
survival probability in each state is given by Eq. ([3]) 
with parameters carrying the appropriate index: Ts,fJ-s 
for S and Tp,iip for P. The spectrum of this type of 
fluctuating signal, as calculated in Refs. 



23|,l2J, is: 
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valid for /i < 2, remarkably, even though a stationary 
auto-correlation function cannot be defined in this case. 
In the case /t > 2, S{f) — A/f^~f^, with A independent 



of L, the length of the sequence under study. Similarly 
to the rate of events Rs (t) the spectral intensity per unit 
time tends to vanish for /i < 2 as an effect of increasing L. 
The ideal 1// noise condition, corresponding to fj, — 2, 
generates instead a logarithmic decrease of the spectral 
intensity with time, and consequently a spectrum virtu- 
ally independent of L. 

Averaging Eq. ([1]) over the external fluctuations ^p (t) 
we obtain: 



{^(t)) = iium 
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As previously mentioned, the NSLRT of Eq. ([T]) rests 
on the preparation of S at time t — 0. We apply the 
same preparation condition to P, thereby generating the 
cascades Rs{t) and Rp{t) described by Eqs. ^ and 
([5]), with the appropriate indexing. Under this condition 
the relaxation of {^p{t)) becomes identical to the survival 
probability ^p (t). Assuming the condition of Eq. ([2]) we 
have the following expression for the average response: 

(a(i)) = e I Rs{t')^s{t - t')^p{t')dt' . (7) 
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The preparation of both S and P makes the average over 
many realizations of the response a(t) to a given stim- 
ulus P vanish for t — ?> oo. While we refer the readers 
to 25] for details, hereby we prove that the intensity of 
the response of S with /is < 2 to ^p{t) does not decay 
if /xp < 2. This is what we mean by going beyond the 
LRT death, claimed by many researchers [l5|, ll7H20J . To 
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TABLE I: Summary of the asymptotic values of the cross- 
correlation function ^{t). * See Eq. ((9|. 

prove this important fact we study the cross-correlation 
(or I/O correlation) function between the system S and 
the stimulus P : C{t) = ((^s (*) Cp (*))) ^nd the mutual 
information, which are used as indicators of aperiodic 
stochastic resonance [27| . Multiplying both sides of Eq. 
© by S,p{t) and averaging over the fluctuations of the 
perturbation P we obtain: 

$(i) = C{t)/e = f dt'Rsit')^s{t - i')*P it, t') ■ (8) 

Note that both Eq. ([7]) and Eq.® depend on the survival 
probability of network P, but the former depends on the 
single time t' whereas the latter depends on both t' and 
t and is non-stationary We limit ourselves to report the 
results for the asymptotic value $c>o of <&(t). When S.si't) 
and S,p{t) are not stationary, i.e. when 1 < /is < 2 and 



1 < ^p < 2, Eq. ([8|, in the limit t -^ oo, gives: 

$oo = C(Ms,Mp) = r(Ais + Aip-2)x (9) 

3-F2 [{fJ.p -l,fip- 1, fJ.p + fis - 2}, {^p, ^p}, 1] 

r(2 - /zp)r(/zp)2r(Ais - 1) 

where 3F2 is the generaUzed hypergeometric function. In 
the case 2 < ^p < 3, $00 is simply zero. 

In the case 2 < //g < 3, inserting into Eq. ^ expres- 
sion ^ for Rs{t), leads to: 
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Eg. pOj) is exact for t ^ ts and for 1 < fip < 2 it leads 
|25| to $00 = 1, while for 2 < ^p < 3 it yields: 

$oo = (Ais-2)/(/ip + /i5-4). (11) 

Results are summarized in Table HI For illustrative pur- 




FIG. 1: The asymptotic limit of "l>(f) is displayed for fis, fJ'P G 
]1, 3[. The vertex fis~fJ.p='2 marks the transition to a condi- 
tion of maximal input-output cross-correlation. 

poses, we supplement Table I with Fig. 1, showing the 
3D plot of the cross-correlation function $00 in the same 
parameter range: Square II and square III correspond 
to the condition of minimal and maximal correlation, re- 
spectively. Intuitively it is so because of the difference of 
time scales between S and P in such regions. In III fluc- 
tuations £,sit) and Cp{t) have a finite and an infinite time 
scale, respectively, thereby allowing £,s{t) to adapt to the 
stimulus-induced bias so as to yield maximal correlation. 
In II the role of the time scales is inverted, the bias in- 
duced by P on the longer (diverging) time scale of the 
process £,s{t) is asymptotically averaged out due to the 
many intervening switching events of ^p(t), producing 
no correlation. The vertex /zg = /ip = 2, representing a 
l//-noise system under the stimulus of a l//-noise per- 
turbation, marks the abrupt transition from vanishing 
(square II) to maximal correlation (III). 



Now let us proceed to the demonstration that the in- 
tensity of the response a{t) to a single realization of the 
stimulus does not decay, if $00 7^ 0. We note that by 
definition, the non-vanishing $00 yields: 

C{t) =J2'^ Pi^s{t)^'\^p{t)=j)p{^p{t)=j) ^ e<^oo, 



where the conditional probability for the occurrence of 
a value of ^s=i= ± 1, given the occurrence of a value 
of (,p—j^ ± 1, has been introduced. We note that for 
t -^ 00, on a time scale such that {Cp{t)) ~ (Cp(0))i^ ''^ 
is a second-order quantity, O(e^), we have p{^p{t)—j) = 
1/2 + 0(£2) and $(i) = $00 + 0{e^). Thus, due to the 
symmetry of the considered dichotomous processes: 
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ptim'p)^p{ut)^^\^pit)=j] 



In the same long-time scale, Ec^. ([T2|) yields: 

(a(t))± =Y,P{^s{t)^^Cpit) = ±l)i ~ ±£$00, (13) 

i 

where the subscript ± indicates the value of £,p at time 
t. Summing Eq. P^ over the two values of ^p, gives 
a total average null response, as expected. But if the 
magnitude \o'{t)\ of the response to a single instance of 
the input S.p(t) is considered instead, its total average is: 



(Hm 
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(14) 



where an equality holds if terms of order 0(£^) are ne- 
glected. Thus when $00 > 0, the response a{t) to a 
single instance of the input ^p{t) does not die out and re- 
mains proportional to the stimulus, no matter how large 
t becomes. Square III is the plateau region of maximal 
cross-correlation and response, together with the degen- 
erate limit case fip = 1 of square I. The term death of 
linear response is appropriate for the vanishing correla- 
tion of square II. The total average response {(T{t)) always 
tends to vanish for t ^ 00 for reasons that do not im- 
ply a lack of response except in the case of square II. 
The ever-lasting response to a single complex stimulus is 
confirmed numerically by Fig. 2, whose insert shows the 
correlation emerging from average over many realizations 
of the stimulus. 

The reason for the striking difference between the re- 
sponse to a harmonic perturbation and the response to 
a non-ergodic stimulus is intimately related to the emer- 
gence of 1// noise and to its spectrum described by Eq. 
© which assigns the weight S{f)/L = l/(/i)3-A'p to 
the spectral component of frequency / of a non-ergodic 
stimulus. As a consequence, the stimulus generates, in 
time, smaller and smaller frequencies /, so as to keep 
l/(/i)^^^^ {i-e. the response intensity to frequency / 
[iq ) finite, thereby yielding Eq. (ITi|) . The death of lin- 
ear response 15|, I17h20| is caused by the fact that stimuli 
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FIG. 2: Response cr(f)(black line) to input ^p{t) (green square 
line) rescaled by e, for fj,s=l-9,Ts=10, fJ.p=1.55,Tp=9. In- 
sert: Average of a{t)£^p{t) over A'^=10* inputs, converging to 
"1? 00=0. 85 as predicted by Eq. ((g]). (Color online) 



with fixed frequencies cannot cope with the decreasing 
frequency of the cascade of events of Eq. (JH) . 

We have afforded a compehing proof that the intensity 
of the single realizations of (j{t), with us < 2, does not 
decay if the perturbation ^p (t) falls in the same complex- 
ity basin (//p < 2). We refer to this phenomen with the 
term complexity management to distinguish it from the 
term complexity matching coined in [28| which implies 
maximum response when fj,s = /ip- 

Now we argue that 1// stimuU generate the maximum 
information transport by the mutual information 

Ht) = J2p^iep)Pties\ep)iogMm'p)/Mes)]- (15) 

ij 

From Eq. (12) in fact, it follows that I{t -^ oo) ~ e^$^ 
and the information transmission rate is obtained by mul- 
tiplying I(t) by the input rate [20|, given by Rp{t). If 
fip > 2, Fig. 1 shows that <I>oo < 1- Although square III 
in Fig. 1 indicates that all stimuli with fip < 2 induce 
maximal correlation, ^p < 2 corresponds to a stimulus 
with decaying events rate (input bits/sec) Rp{t). So even 
if a response is produced in this regime, the rate of infor- 
mation vanishes in time. Only at the crucial condition 
fip = 2, of ideal l//-noise , this algebraic decay becomes 
logarithmic, and, consequently, a steady and maximal 
information transmission rate is achieved. 

Experimental verification either on liquid crystals 12 1 
or on ion channels, whose open/close dynamics has been 
reported to have 1// properties [30|, is desirable. In the 
latter case, using patch-clamp technique, a 1// stimulus 
can be used as a stimulus and the correlation with the 
current output analyzed. 

In conclusion, the NSLRT proposed in this Letter [29[ 
explains not only the mystery of the efhcient transport 
of information emerging from the latest theoretical and 
experimental results in neuro-physiology [2, l5| , but many 
other forms of 1// noise propagations, e.g. why ecologi- 
cal time series tend to exhibit 1// noise if the underlying 



abiotic perturbations are 1// noise 31 1. 
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